Filtering Variational Objectives
نویسندگان
چکیده
When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter’s estimator of the marginal likelihood, the filtering variational objectives (FIVOs). FIVOs take the same arguments as the ELBO, but can exploit a model’s sequential structure to form tighter bounds. We present results that relate the tightness of FIVO’s bound to the variance of the particle filter’s estimator by considering the generic case of bounds defined as log-transformed likelihood estimators. Experimentally, we show that training with FIVO results in substantial improvements over training with the ELBO on sequential data.
منابع مشابه
Variational filtering
This note presents a simple Bayesian filtering scheme, using variational calculus, for inference on the hidden states of dynamic systems. Variational filtering is a stochastic scheme that propagates particles over a changing variational energy landscape, such that their sample density approximates the conditional density of hidden and states and inputs. The key innovation, on which variational ...
متن کاملQuantized Variational Filtering for Bayesian Inference in Wireless Sensor Networks
The primary focus of the chapter is to study the Bayesian inference problem in distributed WSNs with particular emphasis on the trade-off between estimation precision and energy-awareness. We propose a variational approach to approximate the particle distribution to a single Gaussian distribution, while respecting the communication constraints of WSNs. The efficiency of the variational approxim...
متن کاملOptimal sensor and path selection for target tracking in wireless sensor networks
This paper addresses target tracking in wireless sensor networks where the nonlinear observed system is assumed to progress according to a probabilistic state space model. Thus, we propose to improve the use of the quantized variational filtering by jointly selecting the optimal candidate sensor that participates in target localization and its best communication path to the cluster head. In the...
متن کاملPlanar Variational Multi-Scale Modeling
The variational multi-scale (VMS) method for large-eddy simulation (LES) is a promising new approach that employs variational projection to achieve a priori scale separation in lieu of traditional spatial filtering. However, depending on the numerical method used, VMS may not be practicable in all spatial directions. We apply the VMS methodology to a numerical method that does not support expli...
متن کاملOn Noncoercive Variational Inequalities
We consider variational inequalities with different trial and test spaces and a possibly non-coercive bilinear form. Well-posedness is shown under general conditions that are e.g. valid for the space-time variational formulation of parabolic variational inequalities. Moreover, we prove an estimate for the error of a Petrov-Galerkin approximation in terms of the residual. For parabolic variation...
متن کامل